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Views of critical thinking were culled from the literature and developed into a 
scholarship of teaching and learning (SoTL) model that was implemented into the 
Internet course, "The Politics and Psychology of Hatred . " Assessment of student 
course postings demonstrated a strong relationship between interpersonal skills 
(referred to in the curriculum as "course etiquette") and advancement on the levels 

of critical thinking. The implications of these findings are discussed. 

Understanding Critical Thinking 

Hutchings and Shulman (1999) define the scholarship of teaching as based 
in a process of critical questioning and answering. As such, it seems important to 
assess methods by which critical thinking skills can be developed and nurtured in 
students. Theories of critical thinking are numerous in the scholarship of teaching 
and learning literature but, in our experience, few efforts have been made to 
demonstrate how critical thinking can be 

taught in courses. Before outlining the critical Indeed, critical thinking 

thinking model that we developed and use in may be a "buzz phrase" that 

an Internet course on the Politics and many use without truly 
Psychology of Hatred, let us quickly knowing what it is, how to 

summarize a few of the models we drew from foster it, or even how to 

to create the version that we found well-suited measure it or recognize 
for teaching interpersonal and critical thinking when students are (or are 
skills in an internet course (Osborne, Kriese, & not) using it. 

Tobey, 2008). Kuhn (1999) presents a 

developmental model of critical thinking that begins with the question, "do we really 
know what critical thinking is?" Indeed, critical thinking may be a "buzz phrase" 
that many use without truly knowing what it is, how to foster it, or even how to 
measure it or recognize when students are (or are not) using it. This task of 
defining critical thinking must be addressed before any critical thinking framework 
can be integrated into a course. 

Before we can determine how to develop assignments that foster the 
elements of critical thinking, we must understand - really understand - those 
elements. Kuhn (1999) starts with the assumption that critical thinking involves 
cognitive competencies that are meta-knowing. These second-order, meta-knowing 
skills involve an awareness of how self and others "know." Kuhn expands this 
notion further by distinguishing three broad categories of meta-knowing: (1) 
metastrategic, (2) metacognitive, and (3) epistemological (1999). Let us quickly 
define each of these before connecting them to the critical thinking and 
interpersonal skills frameworks constructed for our course. 

The metastrategic way of knowing involves an ability to select and monitor 
the thinking strategies that one uses. With this level, students are encouraged to 
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Students should be able to 
describe their own thoughts 
and thought processes 
without necessarily 
understanding the 
significance or ramifications 
of that knowledge. 


ask what they know and to consider how they know it. It is our experience that 
students are not typically encouraged to consider thinking strategies. In fact, there 
may be a general (unspoken) assumption in teaching that students already know 
"how" and "when" to think. We will revisit this point by articulating the critical and 
interpersonal skills we expect from students and, further, how we describe to 
students how such skills will be factored into the grading. 

Metacognitive knowing, according to Kuhn (1999), operates at the level of 
declarative knowledge. What Kuhn appears to mean by this is "fact"-based 
information. In other words, it is important to provide students with the opportunity 
to "tell you what they know." Students should be able to describe their own 
thoughts and thought processes without 
necessarily understanding the significance or 
ramifications of that knowledge. Before one 
can worry about "how" one knows or the 
impact that knowledge has on others, one 
must first express "what" one knows. We refer 
to this expression of knowledge as recitation. 

The recitation process is described as a 
statement of known facts or opinions. A critical element of this step is to 
acknowledge what aspect(s) of what is being stated are factual (declarative) and 
what is based on opinion. It is not assumed that students will know to separate 
facts from opinions in their own thinking without being told to do so, shown how to 
do so, and being held accountable for doing so. 

Epistemological knowing involves understanding how one's knowledge (and 
what one knows) fits into the broader range of what people know. It involves an 
awareness of how people - in general - know something and how one - individually 
- knows it. Kuhn (1999) argues that meta-knowing is developmental in nature. By 
referring to meta-knowing steps as "developmental," Kuhn (1999) implies an 
interactive (nature and nurture) process. In other words, one's experiences will 
determine the extent to which one is able to progress from metastrategic to 
metacognitive and, eventually, to epistemological knowing. We build upon this 
notion by incorporating both individual and group assignments into our course that 
require students to practice with this progression of critical thinking. Critical 
thinking, in Kuhn's model (1999), does not happen by accident nor will it happen 
without experiences that require one to practice it. 

From Kuhn's theoretical notion of critical thinking, we sought guidance on 
how to break critical thinking down into its component elements - elements that 
could be specifically integrated into a course and assignments within that course. In 
other words, we asked the question, "now that we know the theoretical foundations 
for critical thinking, what would examples of critical thinking look like?" We found 
answers to this question in the works of Paul and Elder (2002), and Smith (2002). 
Paul and Elder (2002) suggest that critical thinking involves integrating one’s 
thoughts, feelings and desires. By understanding the relationships among thoughts, 
feelings and desires, Paul and Elder (2002) assert that we can become routinely 
aware of and able to evaluate our feelings. In this way, feelings can inform our 
thoughts rather than override them. 

In the context of our project, this notion of integrating thoughts, feelings 
and desires provided us with the framework we needed to move from the 
theoretical foundation provided by Kuhn to a step-wise approach we could outline 
for students. What we sought was a method for outlining the progression of critical 
inquiry and thought that we expected from students. We wanted something "visible" 
that could be provided to students and to which their work could be held 
accountable. We turn to that framework in a moment, after briefly considering how 
we might approach measuring whether students have achieved the "critical 
thinkers." 

To address this question, we turned to the work of Randolph Smith. Smith 
(2002) asserts that critical thinkers possess seven characteristics: (1) critical 
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thinkers are flexible - they can tolerate ambiguity and uncertainty, (2) critical 
thinkers identify inherent biases and assumptions, (3) critical thinkers maintain an 
air of skepticism, (4) critical thinkers separate facts from opinions, (5) critical 
thinkers don't oversimplify, (6) critical thinkers use logical inference processes, and 
(7) critical thinkers examine available evidence before drawing conclusions (Smith, 
2002). We perceived Smith's (2002) characteristics as identifiable "markers" we 
could look for in assessing student work. Doherty, Hansen and Kaya (2007) remind 
us, however, that, after we employ these characteristics of critical learning, we need 
to return to them periodically to see if they have become a part of students' regular 
thought patterns. When students make these habits their own, they develop a 
vested interest to make sure that these habits continue. In short, when students 
own critical thinking habits they are more in control of not only when and what they 
think but also how they think (Doherty et al., 

2007). 

Clearly there are many more 
examples of, definitions for, and research 
about critical thinking. However, these four 
methods (Kuhn, 1999; Paul & Elder, 2002; 

Smith, 2002; and Doherty et al., 2007) 
allowed us to: (1) build a theoretical foundation for the kind of thinking we wanted 
to foster in our course, (2) delineate the elements that separate critical thinking 
from other forms of thinking, and (3) build an active framework that could be 
described to students, incorporated into assignments, and built into the assessment 
methods we use for those assignments. In other words, we utilized the four 
methods above to develop: (1) the course, (2) the assignments and, (3) the 
grading system we used such that critical thinking could be outlined, developed and 
demanded from our students. The framework we developed is summarized as 
follows: 

1. Recitation - state known facts or opinions. A critical component of this 
step is to acknowledge what aspect(s) of what is being stated are 
factual and what are based on opinion. 

2. Exploration - analyze the roots of those opinions or facts. This step 
requires digging below the surface of what is believed or known and 
working to discover the elements that have combined to result in that 
fact or that opinion. This involves analysis without an attempt to 
comprehend the impact of those facts or opinions. 

3. Understanding - involves an awareness of other views and a 
comprehension of the difference(s) between one’s own opinion (and 
the facts or other opinions upon which that opinion is based) and the 
opinions of others. To truly "understand" our own opinion in 
relationship to others, we must initiate an active dialogue with the 
other person about his or her opinions and the roots of those opinions. 
In other words, once we become aware of the roots of our own 
opinions, we must understand how to discover the roots of the 
opinions of others. 

4. Appreciation - a full awareness of the differences between our views 
and opinions and those of others. To truly appreciate differences, we 
must be aware of the nature of those differences. The active dialogue 
undertaken in the third step (understanding) should lead to an 
analysis of the opinion as recited by the other. The result should be a 
complete awareness of the similarities and differences between our 
own opinions (and the roots of those opinions) and those of the 
"other." Although we may still be aware that our opinions differ, we 
are now in a position to truly appreciate and value those differences. 


When students own critical 
thinking habits, they are 
more in control of not only 
when and what they think, 
but also how they think. 
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Understanding “Understanding" 


In our view, it is important to acknowledge that "understanding" does not 
mean to "accept." The goal is not to get everyone to agree; the goal is to get people 
to truly explore and understand how and why opinions differ. To understand means 
to realize the circumstances and motivations 
that lead to differences and to realize that 
those differences are meaningful. It is our 
belief that discussing social issues (such as 
prejudice or racism) without requiring 
students to explore the roots of their views, 
understand the roots of other views, and 
appreciate the nature and importance of 

different views about those issues perpetuates ignorance. To raise the issue without 
using a critical thinking framework may simply reinforce prejudices by giving them 
voice without question. Among the various rubrics that can be used to undertake 
such an assessment of critical thinking, Coster and Ledovski's (2005) tool comes to 
mind. They assess students in three categories and rate them from high ability and 
low ability. These categories are: (1) contributes to the discussion, (2) presents 
one's own opinion on the issue and, (3) assesses the quality of support available 
(Coster & Ledovski, 2005). 


To understand means to 
realize the circumstances 
and motivations that lead to 
differences and to realize 
that those differences are 
meaningful. 


Enhancing Critical Thinking 

The process of implementing critical thinking into our course began with 
the theoretical framework already outlined (Kuhn, 1999) and then proceeded to the 
design phase in which we used the work of Paul and Elder (2002), Smith (2002) and 
others to design a critical thinking framework that provided guidance for students 
on the elements of critical thinking and an understanding that these elements build 
upon each other. Finally, we progressed to a stage of course and assignment 
construction. In other words, knowing what critical thinking is, being able to break 
down critical thinking into its component elements, and demonstrating these 
elements to students to prepare students to engage in critical thinking. These skills 
must be practiced within the course and the assignments within that course. So, we 
set about the task of constructing assignments that would require all elements of 
critical thinking; that would foster student growth along this developmental 
continuum of critical thinking (Kuhn, 1999); and that would hold students 
accountable for demonstrating growth along that continuum. 

Before ever teaching the course the first time, however, we knew we 
wanted to teach the course online. The reason for this, initially, was simple: our 
university wanted faculty to develop online courses with consistent content. Our 
early experiences in teaching the course online, however, taught us that the course 
is very well suited to an online format, primarily due to its exploration of sensitive 
and controversial topics (for more discussion of this issue see Osborne, Kriese, & 
Tobey, 2008). While constructing the course site and the assignments, we utilized 
the advice of MacKnight (2000) on how to teach critical thinking skills through 
online discussions. MacKnight (2000) spells out what students and faculty must be 
prepared to do in order to facilitate critical thinking with online discussions. In 
particular, students must have a clear understanding of the assignment and possess 
the social skills necessary to: 

• ask the right questions, 

• listen to each other, 

• take turns and share work, 

• help each other learn, 

• respect each other's ideas, 

• build on each other's ideas, 

• construct their own understanding, and 
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• think in new ways. (MacKnight, 2000, p. 39) 

Additionally, faculty must support disciplined discussions by: 

• maintaining a focused discussion, 

• keeping the discussion intellectually responsible, 

• stimulating the discussion by asking probing questions that hold 
students accountable for their thinking, 

• infusing these questions in the minds of students, 

• encouraging full participation, and 

• periodically summarizing what has or needs to be done (MacKnight, 
2000, p. 39). 

Potts (1994) outlined "best practice" methods for teaching critical thinking. 
Although many overlap with what we have already outlined, one suggestion stood 
out to us as we contemplated building a course and assignments that incorporated 
the best practices we had gleaned from other researchers and teachers. Potts 
(1994) suggests teaching students to build categories. In Potts' own terms, 
"students often are given (and asked to memorize) explicit rules for classifying 
information" (1994, p. 2). But students learn little about how to develop the rules 
themselves when they are provided. Instead, Potts (1994) suggests that students 
be expected to discover the rules needed to build the categories of thinking (and 
knowledge) required in the course. One of the keys to this approach is to assist 
students in this development process without "giving" them the rules. 

Building Interpersonal Skills for Critical Thinking 

Last, we encountered numerous references to interpersonal skills as an 
aspect of critical thinking (Halpern, 1996; Halpern, 1999; Klaczynski, Gordon, & 
Fauth, 1997; MacKnight, 2000; Hansburg & Silberman, 2005). In other words, 
critical thinking is not just a skill that one holds individually. As Gokhale (1995) 
outlined, critical thinking can best be fostered collaboratively. As such, we felt it was 
necessary to combine critical thinking and interpersonal skills training in our course. 
We outlined these skills in the form of course "etiquette" and held students 
accountable for demonstrating these skills so that a safe learning environment was 
maintained in our course. 

In particular, we specifically outline the "interpersonal" expectations for our 
course with this statement: "This is an 

internet course. As such, the success of the 
course relies on active participation by each 
class member throughout the entire semester. 

Even though we are the professors for the 
course, it is designed as a seminar course, 
meaning that active participation from students is essential. Although face-to-face 
interactions will not occur because of our use of the internet, we do expect continual 
communication between members of the class and the course faculty. Even though 
this interaction will be over the Internet, we expect students to use the same 
etiquette that would be used in a classroom during face-to-face interactions. This 
etiquette includes: 

• respect for others (their viewpoints, their values, their beliefs), 

• the right to disagree but requires sensitivity to the viewpoints of 
others, 

• taking responsibility for being involved in developing the issues and 
topics relevant to this course, 

• active participation in all elements of the course, 

• continual feedback to the instructors about the course, course 
assignments, and individual viewpoints, 

• a commitment to the mutual exchange of ideas. This means we will 
not isolate definitive 'answers' to the issues we raise, but we will 
actively explore and respect the multiple sides to those issues, and 

• a responsibility to 'police' ourselves. 
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We are attempting to develop a community, and this requires trust. In order to 
develop trust, we must know that we can share our ideas and not be 'attacked.' 
This also requires that we allow other class members the same trust and freedom 
we expect." 

Putting It All Together - Our Course 

To summarize, here is the sequence we utilized to build our course (and 
the assignments for that course); this sequence reflects a scholarship of teaching 
inquiry model generalizable to other projects and disciplines. We 

1. read much of the critical thinking literature, 

2. delineated what we felt were "best practice” methods for developing a 
model of critical thinking (Paul & Elder, 2002; Smith, 2002), 

3. molded those best practices into expectations for students (Gokhale, 
1995; MacKnight, 2000), 

4. developed methods we would use (as faculty) to encourage critical 
thinking (Gokhale, 1995; MacKnight, 2000), 

5. used those best practices to create individual and collaborative 
assignments (Potts, 1994), and 

6. developed a method for articulating the interpersonal skills necessary 
for critical thinking to our students (Halpern, 1996; Halpern, 1999; 
Klaczynski, Gordon, & Fauth, 1997; Flansburg & Silberman, 2005). 

In structuring the course, we relied on the literature for guidance. We 
started with Gokhale's (1995) work illustrating the impact of collaborative learning 
on critical thinking. Gokhale found that students engaged in collaborative learning 
methods performed better on critical thinking test items than students completing 
the same assignments individually. Following Gokhale's (1995) guidance, we 
decided to develop collaborative assignments for our course. But collaborative 
assignments, while they allow for the assessment of interpersonal skills and critical 
thinking, are not always easy to assess for individual learning. For this reason, both 
individual and collaborative assignments were integrated into the course. 

Our project rested on the idea that the interpersonal skills learned in our 
course would enhance critical thinking and these advancements in critical thinking 
would surely show how students would learn content in future courses. The focus of 
this article, however, is not on student performance (grades) in future courses. Our 
purpose here is to assess the logical but, heretofore, undocumented relationship 
between interpersonal skills and critical thinking discussed throughout the 
scholarship of teaching and learning literature, and to model a scholarship of 
teaching and learning inquiry framework for building a course. 

Method 

Two naive raters went through the course postings from an entire semester 
of the internet- based, team-taught seminar course, "The Politics and Psychology of 
Flatred." Nineteen students participated in the course from the beginning to the end 
of the semester. Student posts were "graded" by faculty but not assessed as part of 
this project until after course grades were submitted. The raters were asked to 
assess the course postings for each student (at the end of the semester) using the 
instruments shown in Appendices A and B. The raters were trained to use the 
scoring systems until the correlation between their scores for a random sample of 
postings pulled from the course reached r=.90. From this point on, the ratings of 
the two raters were averaged for each of the participants in the study. 

As a reminder, our interest was in assessing the relationship between 
interpersonal skills (what we called "course etiquette") and progress on the critical 
thinking model. For our purposes, a course posting that was assessed by raters as 
including "exploration" demonstrated a higher level of critical thinking than one that 
showed "recitation." Our guiding question was: Are higher levels of critical thinking 
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(moving upward through recitation, exploration, and understanding to appreciation) 
related to ratings of student use of the course etiquette (conceptualized by us as 
more effective demonstration of interpersonal skills)? 

Students completed a weekly assignment (responding to a discussion 
forum "question of the week" that was linked to some current event dealing with 
prejudice, discrimination or hate), three individual assignments (larger scale 
assignments to be worked on and posted individually), three group assignments 
(larger scale assignments to be worked on and posted as part of a group - students 
stayed in the same groups all semester), and one course project that was 
completed with the same group as the group assignments. 

Raters (and students at the beginning of the course) were given the 
following definitions to use in assessing course posts for levels of critical thinking: 
(1) Recitation - state known facts or opinions, (2) Exploration - analyze the roots of 
those opinions or facts, (3) Understanding - involves an awareness of other views 
and a comprehension of the difference(s) between one's own opinion (and the facts 
or other opinions upon which that opinion is based) and the opinions of others, and 
(4) Appreciation -a full awareness of the differences between our views and 
opinions and those of others. To truly appreciate differences, we must be aware of 
the nature of those differences. 


Results 


Average rater scores were entered into a linear regression analysis using 
the Statistical Package for the Social Sciences (SPSS) version 11.5. The regression 
analysis was run for each level of the critical thinking model adding in each rating 
on use of course etiquette to determine which elements of course etiquette 
weighted most heavily for each level of critical thinking. As expected, none of the 
levels of course etiquette weighted on recitation of fact and opinion, F(3,8) = 
2.105, p- 292. 


For exploration, several elements of course etiquette had significant beta 
weights and lead to an overall significance of F( 8,3) = 110.632, p=. 001. The course 
etiquette elements that weighted on exploration were respect, sensitivity, and 
mutual exchange of ideas. In terms of the 
third level of the critical thinking model, 
understanding views of others, again there 
was a significant overall effect of etiquette, 

F( 8,3) = 67.646, p=. 003, but the course 
etiquette item that weighted significantly was 
mutual exchange of ideas, t=- 3.667, p=. 035. 

Finally, there was a significant overall relationship between the highest level of 
critical thinking - appreciation of the views of others - and course etiquette, 
F( 8,3) = 908.845, p =. 0001 and the following individual elements of course 
etiquette weighted significantly with appreciation: (1) respect, (2) sensitivity, (3) 
active participation, (4) feedback to others, and (5) mutual exchange of ideas. Each 
of these were significant at the. 05 level. 


Those students who were 
rated most highly on critical 
thinking also demonstrated 
the most use of 
interpersonal skills. 


Discussion 


These findings strongly support the notion spelled out in the literature that 
interpersonal skills are an important element of critical thinking. Indeed, those 
students who were rated most highly on critical thinking also demonstrated the 
most use of interpersonal skills. These findings do not, of course, allow us to 
determine if fostering interpersonal skills enhances critical thinking or if students 
with enhanced critical thinking skills are also more interpersonal. But these data 
demonstrate a clear relationship between the two. In order to determine the order 
of the relationship (which leads to the other), a pre-post design would need to be 
employed. Still, these findings are of value to faculty in terms of demonstrating that 
interpersonal skills are an important element of critical thinking. 
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It is important to describe to students how their work will be assessed for 
critical thinking. To make this clear, we included a clear statement about the critical 
thinking model in our course syllabus. This statement is included in Appendix C. 

The information provided above strongly suggests that: (1) critical thinking 
can be taught in internet courses, (2) interpersonal skills are an important 
component of critical thinking, (3) faculty and students who have not taught or 
taken internet courses have strong differences in perceptions of Internet courses, 
and (4) differences in perceptions between faculty and students about Internet 
courses do not completely go away when faculty have taught such courses and 
students have taken such courses. 

We provide data that interpersonal skills are an important component of 
critical thinking and we outline methods faculty can use to demonstrate, facilitate, 
enhance and assess interpersonal and critical thinking skills in an internet course. 
We believe faculty could incorporate both of these into virtually any Internet course. 
Indeed, it is very important that the reader understand this point very clearly: well- 
designed Internet courses can provide better opportunities to foster the kinds of 
critical thinking processes we have outlined than traditional face-to-face classes. 
Part of this is structural. Because an Internet course does NOT typically involve 
lecturing and presentation of material - material is posted ahead of time for all to 
see and process - the majority of "class time" is spent on reflection and analysis of 
material and student perceptions of that material. 
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Appendix A: Rating of Course Postings 

Overall the posts from this student effectively demonstrates: 

Recitation - state known facts or opinions. 

The posts from this student clearly state known facts or opinions 

1 2 3 4 5 

strongly somewhat neither somewhat strongly 

disagree disagree agree agree agree 

nor disagree 

Exploration - analyze the roots of those opinions or facts. 

The posts from this student effectively explore roots of opinions or facts 

1 2 3 4 

strongly somewhat neither somewhat 

disagree disagree agree agree 

nor disagree 

Understanding - involves an awareness of other views and a comprehension of 

the difference(s) between one's own opinion (and the facts or other opinions upon 

which that opinion is based) and the opinions of others. 

The posts from this student reflect an understanding of the roots of the opinions of 


others. 

9 

^ 



strongly 

disagree 

somewhat 

disagree 

neither 

agree 

nor disagree 

somewhat 

agree 

strongly 

agree 


Appreciation - means a full awareness of the differences between our views and 
opinions and those of others. To truly appreciate differences, we must be aware of 
the nature of those differences. 

The posts from this student reflect an appreciation for the diverse opinions of 


others. 

9 

-3 



strongly 

disagree 

somewhat 

disagree 

neither 

agree 

nor disagree 

somewhat 

agree 

strongly 

agree 


5 

strongly 

agree 
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Appendix B: Ratings of Course Etiquette 


Track the number of times that the student named above, engages in each aspect 
of the course etiquette (put a hash mark for each occurrence of each aspect you 
encounter while reading the posts from this student): 

1. respect for others (their viewpoints, their values, their beliefs), 

2. the right to disagree but requires sensitivity to the viewpoints of 
others, 

3. taking responsibility for being involved in developing the issues and 
topics relevant to this course, 

4. active participation in all elements of the course, 

5. continual feedback to the instructors about the course, course 
assignments, and individual viewpoints, 

6. a commitment to the mutual exchange of ideas. This means we will 
not isolate definitive "answers" to the issues we raise but we will 
actively explore and respect the multiple sides to those issues, and 

7. a responsibility to "police" ourselves. We are attempting to develop a 
community and this requires trust. In order to develop trust, we must 
know that we can share our ideas and not be "attacked." This also 
requires that we allow other class members the same trust and 
freedom we expect. 


Appendix C: Important Notes about Critical Thinking and Course 
Contributions 

All of your written work will be assessed on this model. In other words, all of your 
responses to questions must show all four levels: (1) recitation, (2) exploration, (3) 
understanding, and (4) appreciation. Higher grades will be given to those 
assignments that clearly demonstrate an effort to move upward on this continuum. 

All course contributions (forum postings, exam answers, course papers) must be 
substantive contributions. Substantive contributions are those that demonstrate -. (1) 
that a student has given thought to what he or she has posted, (2) that the 
student's comments have added positively to the discussion, (3) that the 
contributions adhere to the course etiquette principles outlined above, and (4) that 
the student has made progress along the four levels of the critical thinking model. 
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